5,210 research outputs found
Rejoinder: One-step sparse estimates in nonconcave penalized likelihood models
We would like to take this opportunity to thank the discussants for their
thoughtful comments and encouragements on our work [arXiv:0808.1012]. The
discussants raised a number of issues from theoretical as well as computational
perspectives. Our rejoinder will try to provide some insights into these issues
and address specific questions asked by the discussants.Comment: Published in at http://dx.doi.org/10.1214/07-AOS0316REJ the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
The fused Kolmogorov filter: A nonparametric model-free screening method
A new model-free screening method called the fused Kolmogorov filter is
proposed for high-dimensional data analysis. This new method is fully
nonparametric and can work with many types of covariates and response
variables, including continuous, discrete and categorical variables. We apply
the fused Kolmogorov filter to deal with variable screening problems emerging
from a wide range of applications, such as multiclass classification,
nonparametric regression and Poisson regression, among others. It is shown that
the fused Kolmogorov filter enjoys the sure screening property under weak
regularity conditions that are much milder than those required for many
existing nonparametric screening methods. In particular, the fused Kolmogorov
filter can still be powerful when covariates are strongly dependent on each
other. We further demonstrate the superior performance of the fused Kolmogorov
filter over existing screening methods by simulations and real data examples.Comment: Published at http://dx.doi.org/10.1214/14-AOS1303 in the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
On the adaptive elastic-net with a diverging number of parameters
We consider the problem of model selection and estimation in situations where
the number of parameters diverges with the sample size. When the dimension is
high, an ideal method should have the oracle property [J. Amer. Statist. Assoc.
96 (2001) 1348--1360] and [Ann. Statist. 32 (2004) 928--961] which ensures the
optimal large sample performance. Furthermore, the high-dimensionality often
induces the collinearity problem, which should be properly handled by the ideal
method. Many existing variable selection methods fail to achieve both goals
simultaneously. In this paper, we propose the adaptive elastic-net that
combines the strengths of the quadratic regularization and the adaptively
weighted lasso shrinkage. Under weak regularity conditions, we establish the
oracle property of the adaptive elastic-net. We show by simulations that the
adaptive elastic-net deals with the collinearity problem better than the other
oracle-like methods, thus enjoying much improved finite sample performance.Comment: Published in at http://dx.doi.org/10.1214/08-AOS625 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Structured variable selection in support vector machines
When applying the support vector machine (SVM) to high-dimensional
classification problems, we often impose a sparse structure in the SVM to
eliminate the influences of the irrelevant predictors. The lasso and other
variable selection techniques have been successfully used in the SVM to perform
automatic variable selection. In some problems, there is a natural hierarchical
structure among the variables. Thus, in order to have an interpretable SVM
classifier, it is important to respect the heredity principle when enforcing
the sparsity in the SVM. Many variable selection methods, however, do not
respect the heredity principle. In this paper we enforce both sparsity and the
heredity principle in the SVM by using the so-called structured variable
selection (SVS) framework originally proposed in Yuan, Joseph and Zou (2007).
We minimize the empirical hinge loss under a set of linear inequality
constraints and a lasso-type penalty. The solution always obeys the desired
heredity principle and enjoys sparsity. The new SVM classifier can be
efficiently fitted, because the optimization problem is a linear program.
Another contribution of this work is to present a nonparametric extension of
the SVS framework, and we propose nonparametric heredity SVMs. Simulated and
real data are used to illustrate the merits of the proposed method.Comment: Published in at http://dx.doi.org/10.1214/07-EJS125 the Electronic
Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of
Mathematical Statistics (http://www.imstat.org
- …